迄今为止，迄今为止，众所周知，对广泛的互补临床相关任务进行了全面比较了医学图像登记方法。这限制了采用研究进展，以防止竞争方法的公平基准。在过去五年内已经探讨了许多新的学习方法，但优化，建筑或度量战略的问题非常适合仍然是开放的。 Learn2reg涵盖了广泛的解剖学：脑，腹部和胸部，方式：超声波，CT，MRI，群体：患者内部和患者内部和监督水平。我们为3D注册的培训和验证建立了较低的入境障碍，这帮助我们从20多个独特的团队中汇编了65多个单独的方法提交的结果。我们的互补度量集，包括稳健性，准确性，合理性和速度，使得能够独特地位了解当前的医学图像登记现状。进一步分析监督问题的转移性，偏见和重要性，主要是基于深度学习的方法的优越性，并将新的研究方向开放到利用GPU加速的常规优化的混合方法。
translated by 谷歌翻译
非刚性可拉伸结构之间的一致性是计算机视觉中最具挑战性的任务之一，因为不变属性很难定义，并且没有针对真实数据集的标记数据。我们基于规模不变几何形状的光谱域提出了无监督的神经网络体系结构。我们在功能地图体系结构的基础上构建，但是表明，一旦等轴测假设破裂，学习本地功能，直到现在，就还不够。我们证明了使用多个量表不变的几何形状来解决此问题。我们的方法是局部规模变形的不可知论，与现有的光谱最新溶液相比，来自不同域的匹配形状的性能出色。
translated by 谷歌翻译
这项工作提出了一种新的循环架构，可以从图像中提取高频模式并将其重新插入几何特征。此过程允许我们增强一方面捕获精细细节的低成本深度传感器的分辨率，并忠于另一方面的扫描地面真相。我们为深度超分辨率任务以及视觉上有吸引力，增强的3D模型提供了最先进的结果。
translated by 谷歌翻译
通常用于深度学习的陡峭血液算法，使用梯度作为下降方向，或者在使用预处理方向偏移之后。在许多场景中，计算梯度由于复杂或非可微分的成本函数而具有数值困难，特别是奇异点旁边。在这项工作中，我们专注于常见于无监督成本职能的总变化半规范的推导。具体而言，我们在新颖的迭代方案中推导出对硬L1平滑度约束的可分辨率代理，我们称之为成本展开。在培训期间产生更准确的梯度，我们的方法通过改进的收敛来实现给定DNN模型的更精细预测，而无需修改其架构或增加计算复杂性。我们在无监督的光学流任务中展示了我们的方法。在培训众所周知的基线训练期间，更换L1平滑度限制，我们报告了对MPI Sintel和Kitti 2015无监督的光学流量基准的改进结果。特别是，我们报告EPE在封闭像素上减少了高达15.82％的，其中平滑度约束是显性的，使得能够检测更加清晰的运动边缘。
translated by 谷歌翻译
Reading comprehension of legal text can be a particularly challenging task due to the length and complexity of legal clauses and a shortage of expert-annotated datasets. To address this challenge, we introduce the Merger Agreement Understanding Dataset (MAUD), an expert-annotated reading comprehension dataset based on the American Bar Association's 2021 Public Target Deal Points Study, with over 39,000 examples and over 47,000 total annotations. Our fine-tuned Transformer baselines show promising results, with models performing well above random on most questions. However, on a large subset of questions, there is still room for significant improvement. As the only expert-annotated merger agreement dataset, MAUD is valuable as a benchmark for both the legal profession and the NLP community.
translated by 谷歌翻译
We show for the first time that large-scale generative pretrained transformer (GPT) family models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal loss of accuracy. This is achieved via a new pruning method called SparseGPT, specifically designed to work efficiently and accurately on massive GPT-family models. When executing SparseGPT on the largest available open-source models, OPT-175B and BLOOM-176B, we can reach 60% sparsity with negligible increase in perplexity: remarkably, more than 100 billion weights from these models can be ignored at inference time. SparseGPT generalizes to semi-structured (2:4 and 4:8) patterns, and is compatible with weight quantization approaches.
translated by 谷歌翻译
Despite the success of large language models (LLMs) in various natural language processing (NLP) tasks, the stored knowledge in these models may inevitably be incomplete, out-of-date, or incorrect. This motivates the need to utilize external knowledge to assist LLMs. Unfortunately, current methods for incorporating external knowledge often require additional training or fine-tuning, which can be costly and may not be feasible for LLMs. To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the decomposed reasoning steps obtained from the chain-of-thought (CoT) prompting. This lightweight approach does not require additional training or fine-tuning and is not limited by the input length of LLMs. We evaluate the effectiveness of RR through extensive experiments with GPT-3 on three complex reasoning tasks: commonsense reasoning, temporal reasoning, and tabular reasoning. Our results show that RR can produce more faithful explanations and improve the performance of LLMs.
translated by 谷歌翻译
Model quantization enables the deployment of deep neural networks under resource-constrained devices. Vector quantization aims at reducing the model size by indexing model weights with full-precision embeddings, i.e., codewords, while the index needs to be restored to 32-bit during computation. Binary and other low-precision quantization methods can reduce the model size up to 32$\times$, however, at the cost of a considerable accuracy drop. In this paper, we propose an efficient framework for ternary quantization to produce smaller and more accurate compressed models. By integrating hyperspherical learning, pruning and reinitialization, our proposed Hyperspherical Quantization (HQ) method reduces the cosine distance between the full-precision and ternary weights, thus reducing the bias of the straight-through gradient estimator during ternary quantization. Compared with existing work at similar compression levels ($\sim$30$\times$, $\sim$40$\times$), our method significantly improves the test accuracy and reduces the model size.
translated by 谷歌翻译
Most existing pruning works are resource-intensive, requiring retraining or fine-tuning of the pruned models for accuracy. We propose a retraining-free pruning method based on hyperspherical learning and loss penalty terms. The proposed loss penalty term pushes some of the model weights far from zero, while the rest weight values are pushed near zero and can be safely pruned with no need for retraining and a negligible accuracy drop. In addition, our proposed method can instantly recover the accuracy of a pruned model by replacing the pruned values with their mean value. Our method obtains state-of-the-art results in retraining-free pruning and is evaluated on ResNet-18/50 and MobileNetV2 with ImageNet dataset. One can easily get a 50\% pruned ResNet18 model with a 0.47\% accuracy drop. With fine-tuning, the experiment results show that our method can significantly boost the accuracy of the pruned models compared with existing works. For example, the accuracy of a 70\% pruned (except the first convolutional layer) MobileNetV2 model only drops 3.5\%, much less than the 7\% $\sim$ 10\% accuracy drop with conventional methods.
translated by 谷歌翻译
Most of the existing works use projection functions for ternary quantization in discrete space. Scaling factors and thresholds are used in some cases to improve the model accuracy. However, the gradients used for optimization are inaccurate and result in a notable accuracy gap between the full precision and ternary models. To get more accurate gradients, some works gradually increase the discrete portion of the full precision weights in the forward propagation pass, e.g., using temperature-based Sigmoid function. Instead of directly performing ternary quantization in discrete space, we push full precision weights close to ternary ones through regularization term prior to ternary quantization. In addition, inspired by the temperature-based method, we introduce a re-scaling factor to obtain more accurate gradients by simulating the derivatives of Sigmoid function. The experimental results show that our method can significantly improve the accuracy of ternary quantization in both image classification and object detection tasks.
translated by 谷歌翻译